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ABSTRACT 

A new cognitive approach is proposed for understanding the texts of 
planimetric tasks and for visualizing the task conditions to complement the 
syntactic-semantical sentence parsing. Two main difficulties in understanding 
texts of plane geometry tasks are observed: the ellipticity and vagueness of 
texts. To overcome the difficulties in understanding the task conditions it is 
proposed constructing cognitive models of objects and relations between 
them. The proposed cognitive approach is incorporated in an integrated 
system for automatic solving planimetric tasks with the natural language 
interface. The interactive visualization has been developed in the system. It 
depicts the syntactic and semantic structures as a result of natural language 
text analysis and searching for task solution. This visualization allows the 
users to obtain explanations associated with any elements of the images and 
to correct the tasks’ texts in dialog with the system. The destiny of the system 
is to serve for training schoolchildren in the domain of Euclidean geometry. 


cognitive approach, The cognitive approach proposed can be a first step to automated analyzing 
dynamic visualization, plane geometry texts, in perspective, as a cognitively controlled parsing. 
intelligent system. 
© 2020 IJCRSEE. All rights reserved. 
1. INTRODUCTION Drawing the conditions of geometrical 


An integrated intelligent system to 
solve natural language planimetric tasks 
is considered. The system embodies some 
intellectual characters: it contains and uses 
the problem domain knowledge (plane 
geometry), it has a natural language interface 
and understands texts of geometric tasks. The 
solver of the system works based on heuristic 
search for solution. Visualization in this system 
is aimed at showing as much as possible all the 
stages of the system’s functioning and giving 
in the “point and click” manner explanations 
about the content and genesis of any element 
of the drawing. 
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tasks 1s a key problem in the system (for 
students too). The difficulty in analyzing and 
understanding the text is induced by various 
reasons. One reason 1s the ellipticity of the 
text. The resolution of ellipses in the texts 
of planimetric tasks is considered by us in 
(Kurbatov, Naidenova and Ganapol’sku, 
2019). 

However, there are difficulties in 
understanding texts without ellipses too. 
These difficulties are caused, first, a vague text 
language that is not logically and linguistically 
clear. Secondly, these difficulties are induced 
by the necessity to attract general geometric 
knowledge related to objects and relationships 
in the texts of tasks. Thirdly, some difficulties 
are explained by the need to choose from 
several building options, or to formulate 
additional considerations (conditions) for 
drawing. The latter circumstance requires 
the involvement, in the process of drawing, 
various assumptions and logical conclusions. 

An example of a lack of text clarity 
might be the task: “two circles of radii r and 
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R (r < R) are located such that one of their 
internal tangents is perpendicular to one of 
their external tangents. Find the area of the 
triangle formed by these tangents and one of 
the internal tangents”. 

Some examples of involving 
common geometric knowledge are: 

1. “Three circles, the radii of which are 
1, 2, and 3, touch in pairs externally. Calculate 
the radii of two circles, each of which touches 
to three given circles”. (Here it is required 
understanding how to build two additional 
circles). 

2. “Through point R, lying on the 
continuation of diagonal AC of quadrangle 
ABCD and the middle of sides BC and CD, 
are drawn two straight lines crossing sides of 
AB and AD, respectively, in points E and F. 
Prove that the straight lines EF and BD are 
parallel”. (In this task, it 1s necessary to take 
into account that through two points you can 
draw only one straight line, and two straight 
lines should intersect at point R). 

3. “Find the corners of an equilateral 
triangle if its altitude is half the bisector of 
angle at the base”. (Here you have to decide 
what altitude is meant). 

4. “A square is inscribed in the other 
square. Calculate a smaller angle between the 
sides of the squares if their areas are related as 
2:3”. (It is important to consider the position 
of the vertices of the inscribed square). 

There are the tasks for which drawing 
their conditions is possible only after their 
solution. For example: “Is there a rectilinear 
polygon in which the length of one of its 
diagonals equals the sum of two _ other 


the 


diagonals?” 
Call these difficulties cognitive 
expectations. Cognitive expectations are 


apparently quite common when generating 
natural language texts. That 1s why we come 
to the idea of involving in the analysis of texts 
cognitive graphics and relations pre-formed in 
the system of solving planimetric problems. 
We can use them during the visualization of 
a task condition in dialogue with a user. The 
user can be a high school student, a teacher, 
and a schoolboy. 

Despite the fact that the problems 
of resolving ellipses are widely discussed 
theoretically, most of works address only to 
a special type of ellipses, namely the verb 
ellipses (VE) and exclusively for English 
(Kenyen-Dean, Cheung, and Precup, 2016), 
(Liu, Gonzalez and Gillick, 2016), (McShane 
and Babkin, 2015; 2016). These ellipses refer 
to the omission of a verb phrase whose meaning 


can be reconstructed from the context. The 
structure of this ellipsis consists of two parts 
standing in a sentence on the right and left of 
the “dash”. An example: “one had the power 
of the Sun, the other — the Moon’”’. 

To resolve multiple ellipses, a new 
method is advanced in (Shuster, Nivre and 
Manning, 2018). An example of multiple 
ellipsis is: “the prices growth amounted to 
11.9% in 2003, in 2009 — 4.4 %, in 2014 — 
TD 70: 

It should be noted that the question of 
how to restore the full structure of elliptical part 
of a sentence has not been fully solved in the 
conventional approach based on syntactical- 
semantic parsing sentences. Linguists 
have already realized the restriction of the 
syntactical-semantic approach to resolving 
ellipses in which syntax is separated from 
semantics (Jurafsky, 1993, p. 3). In (Zhao, 
2016), the following answers are compared 
to the basic questions in the framework of 
generative linguistics and cognitive approach: 

¢ Is ellipsis a syntax unit? 

¢ Is the meaning of ellipsis determined 
only by its antecedent? 

¢ Is reconstructing the missed part in 
a sentence equivalent to understanding the 
ellipse in it? 

Generative linguists answer these 
questions in the affirmative. Adepts of cognitive 
linguistics tend to answer in the negative. 
Thus, P. W. Culicover and R. Jackendoff, 
(2006, p. 414) state that there 1s no additional 
syntax structure corresponding to the missing 
words in the sentence and, therefore, the study 
of ellipses based on grammatical rules does 
not make sense. 

The cognitive approach states that the 
meaning of the missed part of a sentence 
depends greatly on the meaning of the whole 
sentence. Understanding ellipsis does not 
mean that we first have to restore it, and then 
to turn to understanding the whole sentence. 
In fact, understanding the sentence also entails 
understanding the ellipsis in it. The meaning 
of the ellipsis is defined both by the explicit 
part of a sentence and by the knowledge of the 
subject area, including linguistics, pragmatics, 
encyclopedic knowledge, context etc. 

Generative linguistics is not interested 
in the real human mental processes taking 
place in ellipsis resolutions. Cognitive 
linguists suggest that the design of meaning 
is a conceptual process (Langacker, 2009) and 
language itself does not encode meanings, but 
only gives a hint at their design (Evans and 
Green, 2006, p. 162). 
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2. MATERIALS AND METHODS 


2.1. Description of the system for 
automated solving planimetric tasks 


The main concept of the system for 


developed in (Khakhalin et all., 2012), 
(Kurbatov, Fominykh and Vorobyev, 2019) 
and its general scheme is given in Figure I. 
The concept of “integrated system” covering 
natural language interface, heuristically 
oriented solver and conceptual visualization 1s 
described in (Lobzin et all., 2015). 





solving planimetric problems has_ been 
ONTOLOGY 
— - Linguistic 
hor AROLOD translator — 


rs 
ne 


The translator uses linguistic knowledge 


Graphics + NL _ 


J 


GRF - interpreter 






_ about the natural language 
ae Ee eee 


about the interface 
with graphics 
SA - about subject area 


about the language (morphology and syntax), and knowledge 


about mapping phrases of natural language to ontology. 
The solver uses knowledge of subject area 
(objects, properties, relations, axioms) for 
ontologically-oriented solutions. 
GRF-interpreter provides an interface between 
ontological and graphical objects. 


a 


Figure 1. General scheme of the system 


The system consists of the following 
blocks: “Ontology”, “Solver’, “Linguistic 
translator’, “Graphics+NL”, and “GRF 
interpreter’ modules for drawing and 
explaining the results accompanied by the 
NL-explanation of the solution process. 

The ontology serves for representing 
knowledge necessary for functioning all the 
subsystems of the system. 

The task of the linguistic translator is to 
construct the conceptual description of a given 
geometrical situation in terms of concepts and 
relations of the ontology. 

The solver takes the ontological 
description of task and searches for solution 
modifying the solution’s intermediate semantic 
representation. 

We use the semantic hypergraphs’ 
language for the ontology representation. This 
language is an extension of semantic networks 
and it provides a suitable basis for naturally 
representing n-dimensional relations. For the 
Ontology’s implementation, DBMS Progress 
has been used. In more details, the system 
ontology is described in (Kurbatov and 
Vorobyev, 2016). 

The linguistic translator performs 
several processes: traditional grammar and 


semantic analyses, and semantic interpretation 
of planimetric task texts. The grammar 
analysis covers morphological and syntactic 
analyses. The semantic interpretation 
consists in “translating” text’s fragments into 
corresponding ontological structures. 

Processing NL-texts of planimetric 
tasks 1s based on the linguistic concept of 
paraphrasing (Apresian et all., 2010). With the 
help of the paraphrasing, subject-oriented text 
is translated into canonical structures directly 
displayed in the ontology. In the aspect of our 
fixed subject area, 1t 1s assumed that there are 
some standard (canonical) NL-descriptions of 
objects and relations. 

The solver uses two components: 
heuristic and logical ones. The examples 
of heuristic rules are: reducing a task to its 
algebraic formulation; using the geometric 
concept of “locus of points”; searching for 
some cognate task; focusing on objects having 
maximal number of known and derived 
constituents; using empirical guesses; using 
statistical data; limiting the depth of search; 
beginning with the simple actions; using the 
symmetry; the trial and error method; using 
inductive reasoning, and some others. 

The operations are performed on 
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semantic structures (SemS) (Mel’¢uk, 
2018). There are two kinds of operations: 
basic operations or construction axioms 
and general operations. Examples of basic 
operations: construction of straight line 
passing through two already constructed 
(given) points; construction of circle having 
the center in an already constructed point and 
radius equal to the segment connecting some 
already constructed points; construction of 
the intersection of two already constructed 
figures; selection of an arbitrary set of points 
belonging to one of already constructed figures. 
General operations realize the construction 
implemented with the use of basic operations 
and, possible, other general ones. 

The interactive visualization provides 
the incremental control of syntactic and 
semantic structure formation and displaying 
the process of searching for task’s solution. 
It is always accompanied by the explanation 
of elements of drawings and system solutions. 
The system allows to modify all the graphical 
images with keeping tasks’ conditions. Figures 
2 and 3 show the syntactic and semantic 
structures for the following task: 


Build a circle passing through two given points 
and having the centre on a given straight line 


build 
what? - syntactic relation 





AVING (what?) centre + group 


Semantic structure 


cucle 


are different 





Figure 2. Syntax and semantic structures 


“Build a circle passing through two 
given point and having the center on a given 
straight line”. 

It is possible by clicking on an object, for 
example, on word “build” or link “what?”, to 
obtain the information about the characteristics 


of selected issue and the grounds for its 
creation (appearance). 






Drawing 






type =point 
built (decision!) 


Figure 3. Drawing dynamically formed 
by the system as the solution protocol 


The system visualization 
greatly debugging the ontology. 

Each semantic and syntactic structure 
has its presentation in the ontology. This 
presentation is transformed into the natural 
language text taken by the program of 
visualization. Based on linguistic processing 
and solution search, a protocol is extracted 
from the ontology that forms a text file with a 
visualization program. 

The interactive visualization is 
implemented based on javascript. Libraries 
JSXGraph (JSX Graph Reference) and 
MathJax (MathJax Documentation) were 
used to support graphics and mathematical 
formulas. 


simplifies 


2.2. Cognitive models of objects and 
relations 


The process of binding objects extracted 
from texts can be supported by cognitive 
models of objects and relationships between 
them. Cognitive scheme is designed to make 
syntax analysis of texts more effective, 
especially in cases of ambiguity and ellipses. 
Therefore, the cognitive scheme will combine 
three components: 

¢ Semantic component in the form of a 
specific relationship between objects (typical 
geometric situation); 

¢ Syntactical component associated with 
the semantic component, on the one hand, and 
with the corresponding fragments of text, on 
the other hand; 

¢ Visual component in the form of a 
drawing of the corresponding geometric 
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situation. 
Semantic component can be expressed 
by using the system ontology. 


We also assume _ that cognitive 
structures correspond to profound structures 
of geometrical situations outlined in the texts 
and define the structures of noun phrases 
(NPs), prepositional phrases (PPs), and verb 
phrases (VPs). Cognitive approach deals with 
modeling processes occurring in the human 
brain during solving the complex thought 
problems. In the case of geometric constructs, 
the cognitive process is associated with 
thinking about concrete objects (Sechenov, 
2008). Cognitive models reflect the following 
relationships: 

- object carries out some actions; 

- object is subjected to actions of other 
objects; 

- object has different relationships 
(spatial, temporal) with other objects; 

- object can be compound; 

- object can be a part of another object; 

- object has properties (call them actant 
ones) related to the actions that object carries out 
(intersects — intersecting, touches — touching) 
or the actions performed on it (has been 
constructed — constructed, has been inscribed 
— inscribed). Thus, the actant properties are 
directly reflected in the morphological forms 
of words describing them; 


- the relationships between the properties 
of one object or between the properties of 
different objects. 

The cognitive models of objects and 
actions are created, in our approach, in an 
incremental mode using geometry school 
textbooks. Tables 1 and 2 depict some 
fragments of cognitive model “Bisector’. 
An example of interacting NP and VP with 
cognitive scheme is given in Figure 4. 


3. RESULTS 


3.1. Dynamic visualization of 
cognitive models 


Within the proposed approach, it 1s 
possible to use the tool of interactive dynamic 
visualization to create the initial drawing 
representing the task conditions and implicit 
relationships hidden in them. Now the 
ontology will be involved in constructing the 
draft of task condition. 

Consider the cognitive scheme and 
its visualization for the following task: Two 
circles are internally touch each other in a 
point A. From the center of greater circle, it 
is drawn radius OB touching the smaller in 
point C. 


In triangle ABC there are taken points M, N and P: M and N - on sides AC and BC, P - on line seqment MN. 


Triangle |S figure; segment IS line 


Cognitive scheme 


(In triangle ABC (there are taken points M,N, and P): (M, N - (on sides AC and BC)), (P - (on line segment MN))) 


(In triangle ABC (there are taken point M, N on sides AC and BC)) 
(in triangle ABC (there is taken point P on line segment MN)) 
(in triangle ABC (there is taken point M on AC)) 
(in triangle ABC (there is taken point N on BC)) 
Due to the connection between NPs via points M.N, and P 
and the coordination with the cognitive scheme, 
it is possible to delete the dash and construct 3 sub-sentences of the given sentence 





Figure 4. An example of interacting VP and NPs with a cognitive scheme 
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Table 1. Noun Phrases including “Bisector”’ 


Bisector Hyperlink to object (to NP) Hyperlink to object (to PP) 
Bisector of angle 

Bisector of angle in (of) triangle 

Bisector of acute angle in (of) rectangular triangle 
Bisector of inner angle in (of) triangle 

Bisector of angle at base of isosceles triangle 
Bisector coming from vertex of inscribed triangle 
Bisector of angles adjacent to one side in (of) parallelogram 


Bisector of 


Bisector of inner angle 
Bisector of angle 
Bisector of angle 


Table 2. Verb Phrases _ including 
“Bisector” 
Hyperlink to Hyperlink to object 
Bisector 
action (to PP) 
Dividing To divide Side of triangle 
Perpendicular Tobe perpendicu- Median of triangle 


Splitting. lar Side of parallelogram 


cutting in To split, to cutin in segments 


Intersecting To intersect Bisector of triangle 
Intersecting To intersect Circle 
Restricting To restrict Area of quadrangle 


Coming across To come across Circle in points 


Containing To contain Points of intersection 
Lying on To lie Straight line 
We need the cognitive scheme 


“internally touching two circles” (Figure 5). 
The dynamic visualization program generates 
all possible variants of this situation, but it 
keeps the condition of “internally touching” 
(as an invariant one). The ratio between the 
lengths of the circles’ radii and the reciprocal 
positions of their centers will be changeable. 
The center of the larger circle may be outside 
or within the smaller circle. So, we have two 
possible drawing of the current situation. 
Now we take into account the other 
condition of the task: from the center of 
greater circle, it is drawn radius OB touching 
the smaller in point C. We turn to the cognitive 
scheme “to draw a tangent to a circle” (Figure 


in (of) triangle 
in (of) parallelogram 
in (of) convex quadrilateral 


in (of) rectangle 


6). From this scheme, it will be known that 
the tangent is located outside the circle and 
has only one common point with it. Since the 
tangent line is simultaneously the radius of 
the larger circle, this radius should lie outside 
the smaller circle. Therefore, the center of the 
larger circle should lie outside the smaller 
circle. We have two variants of drawing 
consistent with the previous conclusion. Since 
there are no more conditions limiting the 
choice, we can take any option. Then the angle 
is selected the magnitude of which is unknown 
(Figure 7). 

In the process of text analysis, the 
drawing satisfying a task conditions is created 
incrementally. This means that the solution of 
tasks begins together with this process. 

The use of an interactive cognitive- 
oriented visualization is reflected in Figures 5, 
6, and 7. After the "Start building" step, the 
user can modify the drawing (moving elements 
by mouse or requesting a system modification 
of the drawing). However, this action requires 
the organization of a dialogue with the system 
in the process of work. 
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Start building || Continue to build-1 || Continue to build-2 || Continue to build-3 || show BAC || decision | 





Two circles touch each other internally at point A. 
From the center of the larger circle, the radius of the OB is drawn, 
which touches the smaller one at point C. Find the angle BAC. 


Two circles are constructed 
touching internally at the point A 





Figure 5. The draw of task after “Start building” (screen shot step) 


OB radius built 





Figure 7. The draw of task after the step 
“Decision” 


Figure 6. The draw of task after the : ee : 
step “Continue build-2” 3.2. Using cognitive schemes in 


dialog with the users 


Let a task be given: "two circles of radii 
r and R (r < R) are located such that one of 
their internal tangents is perpendicular to one 
of their external tangents. Find the area of the 
triangle formed by these tangents and one of 
the internal tangents”. Figure 8 is the drawing 
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corresponding with this task. 
Suppose that the system fails to 
understand completely the text of this task and 


cannot construct the correct drawing of the 
task condition. 





Figure 8. Drawing the initial cognitive structure 


The proposed solution is advanced as 
follow: 

1. The task text is used for extracting 
available cognitive model (for example, via 
the key words); 

2. This model includes: 

* semantic cognitive structure; 

¢the drawing of cognitive semantic 
structure; 

¢ the natural language description of this 
drawing. 

The drawing of cognitive structure 
(scheme) is showed to the user via a dialog 
(by means of simple commands (operations)). 
The user changes the drawing. The cognitive 
semantic structure and the task text are 
changes, 1n parallel, automatically. 

After entering the changes, the system 
shows the new formed task text to the user. 

If the user supports the text, then the new 
cognitive semantic structure is transmitted to 
the system solver. 


The initial cognitive semantic structure 
is described as follows: 

circle crcl -1 has radius variable r 

circle crcl -2 has radius variable R 

variable r smaller than variable R 


circle crcl-1 has external tangent line 
circle crcl--1 has external tangent line 
pr-2 


circle crcl--2 has external tangent line 


circle crcl--2 has external tangent line 
pr-2 


circle crcl--1 has intenal_ tangent line 


circle crcl--2 has internal tangent line 


poin L on 
point L Ha 


line pr-1 
line pr-2 





Figure 9. The drawing of the cognitive structure after applying the command about 


perpendicular lines 


www.ljcrsee.com 


oe 


Naidenova X., Kurbatov S., & Ganapolsky V. (2020). Cognitive models in planimetric task text processing, 
International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 8(1), 25-35 


Natural language description of the 
drawing: “two circles of radii r and R (r < R) 
have internal and external tangents”’. 

With the use of the command “selected 
straight lines are perpendicular” the drawing 
is changed (it 1s really working example). The 
result is in Figure 9. 

With the use of the command (operation) 









«Find the area of triangle IJK” the task 
condition is made more exact. 

Selecting objects can be done by a click 
of the mouse. 

For example, the click on segment 
DIE] gives some message and the segment 1s 
included in the list of selected objects (Figure 
10). 





Object D1E1 is selected 
Stepp 33: segment D1E1 
is constructed 

second intemaltangent 









Ok 





Figure 10. The message appears after selecting segment DIE] 


Obtaining messages via the click is 
only particular case of the click-calling 
functions. These functions can provide a lot 
of possibilities: dialog, appeal to the ontology, 
editing the knowledge. 

After supporting the cognitive scheme 
by the user, the initial text of cognitive scheme 
"two circle of radii r and R (r< R) have internal 
and external tangents” is completed by the 
following text: 

“one of the internal tangents is 
perpendicular to the external one. Find the area 
of triangle formed by the internal tangents and 
this external tangent”. 

And after supporting the changed text 
of cognitive scheme by the user, the semantic 
structure is completed by the following 
fragment: 


line pr-1 perpendicular line pr-3 
triangle IJK has the area ? 


And finally, the changed semantic 
structure is passed to the solver. 

The commands form the main objects 
(points, straight lines, circles...) and the 
relationships between them (to belong, to 
be perpendicular, to be parallel, to intersect, 
...). These operations call directly the library 
functions jsxgraph as well as create the 
commenting arrays (step descriptions and 


their explanations). 

Of course, a step-by-step view of 
forming the drawing included in a cognitive 
structure is also provided, as well as a number 
of service functions - "freeze" the drawing, 
viewing auxiliary constructions (invisible on 
the drawing to avoid bulkiness), etc. 

For example, you can make visible all 
auxiliary builds, here's an example (Figure 
11): In the process of text analysis, the 
drawing satisfying a task condition 1s created 
incrementally. 

The use of an interactive cognitive- 
oriented visualization is reflected in Figures 
8-10. The user can modify the drawing 
(moving elements by mouse or requesting 
a system modification of the drawing). This 
action means the organization of a dialogue 
with the system in the process of work. 
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Figure 11. Auxiliary straights and circles needed to build tangents using known algorithms 


4. DISCUSSIONS 


We resume the features of cognitive 
models as follows: 

¢ Each cognitive model (scheme) can be 
displayed as a drawing; 

¢ Visualized dynamic cognitive model 
determines invariant relationships between 
its objects as like as all their possible 
configurations; 

¢ Visualized cognitive models allow to 
list all the implied objects and relationships 
between them and to restore elided or hidden 
elements in the task text; 

¢ The interaction of cognitive schemes 
implies narrowing the search for task solution. 

Tuning system parameters, modifying 
heuristics and eliminating explicit defects are 
carried constantly. 

Introducing the concept of cognitive 
model of geometry objects and actions 
implies that the structure of automated 
analysis of geometric texts can be considered, 
in perspective, as a cognitively controlled 
parsing. 

The cognitive models imply also the 
possibility to synthesize a text describing a 
geometric situation and compare this text with 
the text to be analyzed. 


5. CONCLUSIONS 


The essential role of cognitive approach 
for understanding natural language texts 
of planimetric tasks and displaying task 
conditions in the form of a drawing is shown. 


A structure of cognitive schemes to represent 
planimetric objects, relations between them 
and planimetric constructions is proposed. 
Two kinds of difficulties in understanding the 
task texts are described: the ellipticity of texts 
and their vagueness. It is proposed the dynamic 
visualization of cognitive schemes and using 
them in dialog with the user to create the draft 
of task condition and to restore the task text. 

The organization of natural language 
dialog between the system and the users at all 
the steps of the system’s functioning 1s the aim 
of our future investigations. 
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